Do machine learning methods lead to similar individualized treatment rules? A comparison study on real data.

Identifying patients who benefit from a treatment is a key aspect of personalized medicine, which allows the development of individualized treatment rules (ITRs). Many machine learning methods have been proposed to create such rules. However, to what extent the methods lead to similar ITRs, that is, recommending the same treatment for the same individuals is unclear. In this work, we compared 22 of the most common approaches in two randomized control trials. Two classes of methods can be distinguished. The first class of methods relies on predicting individualized treatment effects from which an ITR is derived by recommending the treatment evaluated to the individuals with a predicted benefit. In the second class, methods directly estimate the ITR without estimating individualized treatment effects. For each trial, the performance of ITRs was assessed by various metrics, and the pairwise agreement between all ITRs was also calculated. Results showed that the ITRs obtained via the different methods generally had considerable disagreements regarding the patients to be treated. A better concordance was found among akin methods. Overall, when evaluating the performance of ITRs in a validation sample, all methods produced ITRs with limited performance, suggesting a high potential for optimism. For non-parametric methods, this optimism was likely due to overfitting. The different methods do not lead to similar ITRs and are therefore not interchangeable. The choice of the method strongly influences for which patients a certain treatment is recommended, drawing some concerns about their practical use.

Strong instrumental variables biased propensity scores in comparative effectiveness research: A case study in oncology.

BACKGROUND AND OBJECTIVES: Some medications require specific medical procedures in the weeks before their start. Such procedures may meet the definition of instrumental variables (IVs). We examined how they may influence treatment effect estimation in propensity score (PS)-adjusted comparative studies, and how to remedy. STUDY DESIGN AND SETTING: Different covariate assessment periods (CAPs) did and did not include the month preceding treatment start were used to compute PS in the French claims database (Sytème National des Données de Santé-SNDS), and 1:1 match patients with metastatic castration resistant prostate cancer initiating abiraterone acetate or docetaxel. The 36-month survival was assessed. RESULTS: Among 1, 213 docetaxel and 2, 442 abiraterone initiators, the PS distribution resulting from the CAP [-12; 0 months] distinctly separated populations (c = 0.93; 273 matched pairs). The CAPs [-12;-1 months] identified 765 pairs (c = 0.81). Strong docetaxel treatment predictors during the month before treatment start were implantable delivery systems (1% vs. 59%), which fulfilled IV conditions. The 36-month survival was not meaningfully different under the [-12; 0 months] CAP but differed by 10% points (38% vs. 28%) after excluding month -1. CONCLUSION: In the setting of highly predictive pretreatment procedures, excluding the immediate pre-exposure time from the CAP will reduce the risk of including potential IVs in PS models and may reduce bias.

Propensity score matching after multiple imputation when a confounder has missing data.

One of the main challenges when using observational data for causal inference is the presence of confounding. A classic approach to account for confounding is the use of propensity score techniques that provide consistent estimators of the causal treatment effect under four common identifiability assumptions for causal effects, including that of no unmeasured confounding. Propensity score matching is a very popular approach which, in its simplest form, involves matching each treated patient to an untreated patient with a similar estimated propensity score, that is, probability of receiving the treatment. The treatment effect can then be estimated by comparing treated and untreated patients within the matched dataset. When missing data arises, a popular approach is to apply multiple imputation to handle the missingness. The combination of propensity score matching and multiple imputation is increasingly applied in practice. However, in this article we demonstrate that combining multiple imputation and propensity score matching can lead to over-coverage of the confidence interval for the treatment effect estimate. We explore the cause of this over-coverage and we evaluate, in this context, the performance of a correction to Rubin's rules for multiple imputation proposed by finding that this correction removes the over-coverage.

Open science saves lives: lessons from the COVID-19 pandemic.

In the last decade Open Science principles have been successfully advocated for and are being slowly adopted in different research communities. In response to the COVID-19 pandemic many publishers and researchers have sped up their adoption of Open Science practices, sometimes embracing them fully and sometimes partially or in a sub-optimal manner. In this article, we express concerns about the violation of some of the Open Science principles and its potential impact on the quality of research output. We provide evidence of the misuses of these principles at different stages of the scientific process. We call for a wider adoption of Open Science practices in the hope that this work will encourage a broader endorsement of Open Science principles and serve as a reminder that science should always be a rigorous process, reliable and transparent, especially in the context of a pandemic where research findings are being translated into practice even more rapidly. We provide all data and scripts at https://osf.io/renxy/ .

A curvilinear bivariate random changepoint model to assess temporal order of markers.

In biomedical research, various longitudinal markers measuring different quantities are often collected over time. For example, repeated measures of psychometric scores are very informative about the degradation process toward dementia. These trajectories are generally nonlinear with an acceleration of the decline a few years before the diagnosis and a large heterogeneity between psychometric tests depending on the underlying cognitive function to be evaluated and the metrological properties of the test. Comparing the times of acceleration of the decline before diagnosis between cognitive tests is useful to better understand the natural history of the disease. Our objective is to propose a bivariate random changepoint model that allows for the comparison of the mean time of change between two markers. A frequentist approach is proposed that gives validated statistical tests to assess the temporal order of the changepoints. Using a spline transformation function, the model is designed to handle non-Gaussian data, that are common for cognitive scores which frequently exhibit a strong ceiling effect. The procedure is assessed through a simulation study and applied to a French cohort of elderly to identify the order of the decline of several cognitive scores. The whole methodology has been implemented in a R package freely available.

Long term evolution of renal function in essential hypertensive patients with no baseline proteinuria.

Data on the long term evolution of renal function in essential hypertensive patients are scarce, showing a low incidence of end stage renal diseases but without information on how the renal function evolves. Our aim is to describe the long term evolution of renal function and possible trajectories in hypertensive patients. We included patients from an ongoing cohort with essential hypertension, no proteinuria at baseline and no diabetes during follow-up and with at least two creatinine dosages 4 years apart. A long term (average 16 years) follow-up was available in 609 patients (baseline age 51.8 ± 11.1 years, 52 % male, mean office BP 156//95 mmHg). The trajectories of creatinine were modeled through a flexible latent class mixed model. The analysis identified three classes of significantly different trajectories. In the first (n = 560), there was no significant variation of creatinine over time. In the second (n = 40), there was a significant rise of creatinine (117 ± 20 vs 85 ± 17 µmol/l, p < 0.0001). The third class (n = 9) was very heterogeneous, mainly composed of outliers. Further analysis showed the nonlinearity of the evolution of creatinine in classes 2 and 3. So the model of progressive renal deterioration in essential hypertension does not fit with our results. A large majority (92%) of patients show no significant change in creatinine level with time. In the others 8%, the increase in creatinine is not progressive but conversely show one or more sudden bouts of elevation.

A hypothesis testing procedure for random changepoint mixed models.

In biomedical research, random changepoint mixed models are used to take into account an individual breakpoint in a biomarker trajectory. This may be observed in the cognitive decline measured by psychometric tests in the prediagnosis phase of Alzheimer's disease. The existence, intensity and duration of this accelerated decline can depend on individual characteristics. The main objective of our work is to propose inferential methods to assess the existence of this phase of accelerated decline, ie, the existence of a random changepoint. To do so, we use a mixed model with two linear phases and test the nullity of the parameter measuring the difference of slopes between the two phases. Because we face the issue of nuisance parameters being unidentifiable under the null hypothesis, the supremum of the classic score test statistic on these parameters is used. The asymptotic distribution of the supremum under the null is approached with a perturbation method based on the multiplier bootstrap. The performance of our testing procedure is assessed via simulations and the test is applied to the French cohort PAQUID of elderly subjects to study the shape of the prediagnosis decline according to educational level. The test is significant for both educational levels and the estimated trajectories confirmed that educational level is a good marker for cognitive reserve.